Stepped-wedge Cluster-randomized Trials Research Articles

Power calculation for detecting interaction effect in cross-sectional stepped-wedge cluster randomized trials: an important tool for disparity research

BackgroundThe stepped-wedge cluster randomized trial (SW-CRT) design has become popular in healthcare research. It is an appealing alternative to traditional cluster randomized trials (CRTs) since the burden of logistical issues and ethical problems can be reduced. Several approaches for sample size determination for the overall treatment effect in the SW-CRT have been proposed. However, in certain situations we are interested in examining the heterogeneity in treatment effect (HTE) between groups instead. This is equivalent to testing the interaction effect. An important example includes the aim to reduce racial disparities through healthcare delivery interventions, where the focus is the interaction between the intervention and race. Sample size determination and power calculation for detecting an interaction effect between the intervention status variable and a key covariate in the SW-CRT study has not been proposed yet for binary outcomes.MethodsWe utilize the generalized estimating equation (GEE) method for detecting the heterogeneity in treatment effect (HTE). The variance of the estimated interaction effect is approximated based on the GEE method for the marginal models. The power is calculated based on the two-sided Wald test. The Kauermann and Carroll (KC) and the Mancl and DeRouen (MD) methods along with GEE (GEE-KC and GEE-MD) are considered as bias-correction methods.ResultsAmong three approaches, GEE has the largest simulated power and GEE-MD has the smallest simulated power. Given cluster size of 120, GEE has over 80% statistical power. When we have a balanced binary covariate (50%), simulated power increases compared to an unbalanced binary covariate (30%). With intermediate effect size of HTE, only cluster sizes of 100 and 120 have more than 80% power using GEE for both correlation structures. With large effect size of HTE, when cluster size is at least 60, all three approaches have more than 80% power. When we compare an increase in cluster size and increase in the number of clusters based on simulated power, the latter has a slight gain in power. When the cluster size changes from 20 to 40 with 20 clusters, power increases from 53.1% to 82.1% for GEE; 50.6% to 79.7% for GEE-KC; and 48.1% to 77.1% for GEE-MD. When the number of clusters changes from 20 to 40 with cluster size of 20, power increases from 53.1% to 82.1% for GEE; 50.6% to 81% for GEE-KC; and 48.1% to 79.8% for GEE-MD.ConclusionsWe propose three approaches for cluster size determination given the number of clusters for detecting the interaction effect in SW-CRT. GEE and GEE-KC have reasonable operating characteristics for both intermediate and large effect size of HTE.

Adherence to key recommendations for design and analysis of stepped-wedge cluster randomized trials: A review of trials published 2016-2022.

The stepped-wedge cluster randomized trial (SW-CRT), in which clusters are randomized to a time at which they will transition to the intervention condition - rather than a trial arm - is a relatively new design. SW-CRTs have additional design and analytical considerations compared to conventional parallel arm trials. To inform future methodological development, including guidance for trialists and the selection of parameters for statistical simulation studies, we conducted a review of recently published SW-CRTs. Specific objectives were to describe (1) the types of designs used in practice, (2) adherence to key requirements for statistical analysis, and (3) practices around covariate adjustment. We also examined changes in adherence over time and by journal impact factor. We used electronic searches to identify primary reports of SW-CRTs published 2016-2022. Two reviewers extracted information from each trial report and its protocol, if available, and resolved disagreements through discussion. We identified 160 eligible trials, randomizing a median (Q1-Q3) of 11 (8-18) clusters to 5 (4-7) sequences. The majority (122, 76%) were cross-sectional (almost all with continuous recruitment), 23 (14%) were closed cohorts and 15 (9%) open cohorts. Many trials had complex design features such as multiple or multivariate primary outcomes (50, 31%) or time-dependent repeated measures (27, 22%). The most common type of primary outcome was binary (51%); continuous outcomes were less common (26%). The most frequently used method of analysis was a generalized linear mixed model (112, 70%); generalized estimating equations were used less frequently (12, 8%). Among 142 trials with fewer than 40 clusters, only 9 (6%) reported using methods appropriate for a small number of clusters. Statistical analyses clearly adjusted for time effects in 119 (74%), for within-cluster correlations in 132 (83%), and for distinct between-period correlations in 13 (8%). Covariates were included in the primary analysis of the primary outcome in 82 (51%) and were most often individual-level covariates; however, clear and complete pre-specification of covariates was uncommon. Adherence to some key methodological requirements (adjusting for time effects, accounting for within-period correlation) was higher among trials published in higher versus lower impact factor journals. Substantial improvements over time were not observed although a slight improvement was observed in the proportion accounting for a distinct between-period correlation. Future methods development should prioritize methods for SW-CRTs with binary or time-to-event outcomes, small numbers of clusters, continuous recruitment designs, multivariate outcomes, or time-dependent repeated measures. Trialists, journal editors, and peer reviewers should be aware that SW-CRTs have additional methodological requirements over parallel arm designs including the need to account for period effects as well as complex intracluster correlations.

The unrecognized role of fidelity in effectiveness-implementation hybrid trials: simulation study and guidance for implementation researchers

BackgroundEffectiveness-implementation hybrid designs are a relatively new approach to evaluate efficacious interventions in real-world settings while concurrently gathering information on the implementation. Intervention fidelity can significantly influence the effectiveness of an intervention during implementation. However little guidance exists for applied researchers conducting effectiveness-implementation hybrid trials regarding the impact of fidelity on intervention effects and power.MethodsWe conducted a simulation study based on parameters from a clinical example study. For the simulation, we explored parallel and stepped-wedge cluster randomized trials (CRTs) and hypothetical patterns of fidelity increase during implementation: slow, linear, and fast. Based on fixed design parameters, i.e., the number of clusters (C = 6), time points (T = 7), and patients per cluster (n = 10) we used linear mixed models to estimate the intervention effect and calculated the power for different fidelity patterns. Further, we conducted a sensitivity analysis to compare outcomes based on different assumptions for the intracluster-correlation coefficient and the cluster size.ResultsEnsuring high fidelity from the beginning is central to achieve accurate intervention effect estimates in stepped-wedge and parallel CRTs. The importance of high fidelity in the earlier stages is more emphasized in stepped-wedge designs than in parallel CRTs. In contrast, if the increase of fidelity is too slow despite relatively high starting levels, the study will likely be underpowered and the intervention effect estimates will also be biased. This effect is more accentuated in parallel CRTs, here reaching 100% fidelity within the next measurement points is crucial.ConclusionsThis study discusses the importance of intervention fidelity for the study`s power and highlights different recommendations to deal with low fidelity in parallel and stepped-wedge CRTs from a design perspective. Applied researchers should consider the detrimental effect of low fidelity in their evaluation design. Overall, there are fewer options to adjust the trial design after the fact in parallel CRT as compared to stepped-wedge CRTs. Particular emphasis should be placed on the selection of contextually relevant implementation strategies.

Power analyses for stepped wedge designs with multivariate continuous outcomes.

Multivariate outcomes are common in pragmatic cluster randomized trials. While sample size calculation procedures for multivariate outcomes exist under parallel assignment, none have been developed for a stepped wedge design. In this article, we present computationally efficient power and sample size procedures for stepped wedge cluster randomized trials (SW-CRTs) with multivariate outcomes that differentiate the within-period and between-period intracluster correlation coefficients (ICCs). Under a multivariate linear mixed model, we derive the joint distribution of the intervention test statistics which can be used for determining power under different hypotheses and provide an example using the commonly utilized intersection-union test for co-primary outcomes. Simplifications under a common treatment effect and common ICCs across endpoints and an extension to closed-cohort designs are also provided. Finally, under the common ICC across endpoints assumption, we formally prove that the multivariate linear mixed model leads to a more efficient treatment effect estimator compared to the univariate linear mixed model, providing a rigorous justification on the use of the former with multivariate outcomes. We illustrate application of the proposed methods using data from an existing SW-CRT and present extensive simulations to validate the methods.

Power swgee: GEE-based power calculations in stepped wedge cluster randomized trials.

Stepped wedge cluster randomized trials are increasingly being used to evaluate interventions in medical, public health, educational, and social science contexts. With the longitudinal and crossover nature of a SW-CRT, complex analysis techniques are often needed which makes appropriately powering SW-CRTs challenging. In this paper, we introduce a newly-developed SW-CRT power calculator, embedded within the power command in Stata. The power calculator assumes a marginal model (i.e., generalized estimating equations [GEE]) for the primary analysis of SW-CRTs, for which other currently available SW-CRT power calculators may not be suitable. The program accommodates complete cross-sectional and closed-cohort designs, and includes multilevel correlation structures appropriate for such designs. We discuss the methods and formulae underlying our SW-CRT calculator, and provide illustrative examples of the use of power swgee. We provide suggestions about the choice of parameters in power swgee, and conclude by discussing areas of future research which may improve the program.

Influence of cluster-period cells in stepped wedge cluster randomized trials.

Stepped wedge cluster randomized trials (SWCRT) are increasingly used for the evaluation of complex interventions in health services research. They randomly allocate treatments to clusters that switch to intervention under investigation at variable time points without returning to control condition. The resulting unbalanced allocation over time periods and the uncertainty about the underlying correlation structures at cluster-level renders designing and analyzing SWCRTs a challenge. Adjusting for time trends is recommended, appropriate parameterizations depend on the particular context. For sample size calculation, the covariance structure and covariance parameters are usually assumed to be known. These assumptions greatly affect the influence single cluster-period cells have on the effect estimate. Thus, it is important to understand how cluster-period cells contribute to the treatment effect estimate. We therefore discuss two measures of cell influence. These are functions of the design characteristics and covariance structure only and can thus be calculated at the planning stage: the coefficient matrix as discussed by Matthews and Forbes and information content (IC) as introduced by Kasza and Forbes. The main result is a new formula for IC that is more general and computationally more efficient. The formula applies to any generalized least squares estimator, especially for any type of time trend adjustment or nonblock diagonal matrices. We further show a functional relationship between IC and the coefficient matrix. We give two examples that tie in with current literature. All discussed tools and methods are implemented in the R package SteppedPower.

A hybrid approach to comparing parallel-group and stepped-wedge cluster-randomized trials with a continuous primary outcome when there is uncertainty in the intra-cluster correlation.

To evaluate how uncertainty in the intra-cluster correlation impacts whether a parallel-group or stepped-wedge cluster-randomized trial design is more efficient in terms of the required sample size, in the case of cross-sectional stepped-wedge cluster-randomized trials and continuous outcome data. We motivate our work by reviewing how the intra-cluster correlation and standard deviation were justified in 54 health technology assessment reports on cluster-randomized trials. To enable uncertainty at the design stage to be incorporated into the design specification, we then describe how sample size calculation can be performed for cluster- randomized trials in the 'hybrid' framework, which places priors on design parameters and controls the expected power in place of the conventional frequentist power. Comparison of the parallel-group and stepped-wedge cluster-randomized trial designs is conducted by placing Beta and truncated Normal priors on the intra-cluster correlation, and a Gamma prior on the standard deviation. Many Health Technology Assessment reports did not adhere to the Consolidated Standards of Reporting Trials guideline of indicating the uncertainty around the assumed intra-cluster correlation, while others did not justify the assumed intra-cluster correlation or standard deviation. Even for a prior intra-cluster correlation distribution with a small mode, moderate prior densities on high intra-cluster correlation values can lead to a stepped-wedge cluster-randomized trial being more efficient because of the degree to which a stepped-wedge cluster-randomized trial is more efficient for high intra-cluster correlations. With careful specification of the priors, the designs in the hybrid framework can become more robust to, for example, an unexpectedly large value of the outcome variance. When there is difficulty obtaining a reliable value for the intra-cluster correlation to assume at the design stage, the proposed methodology offers an appealing approach to sample size calculation. Often, uncertainty in the intra-cluster correlation will mean a stepped-wedge cluster-randomized trial is more efficient than a parallel-group cluster-randomized trial design.

Considerations Before Selecting a Stepped-Wedge Cluster Randomized Trial Design for a Practice Improvement Study.

Despite the growing popularity of stepped-wedge cluster randomized trials (SW-CRTs) for practice-based research, the design's advantages and challenges are not well documented. The objective of this study was to identify the advantages and challenges of the SW-CRT design for large-scale intervention implementations in primary care settings. The EvidenceNOW: Advancing Heart Health initiative, funded by the Agency for Healthcare Research and Quality, included a large collection of SW-CRTs. We conducted qualitative interviews with 17 key informants from EvidenceNOW grantees to identify the advantages and challenges of using SW-CRT design. All interviewees reported that SW-CRT can be an effective study design for large-scale intervention implementations. Advantages included (1) incentivized recruitment, (2) staggered resource allocation, and (3) statistical power. Challenges included (1) time-sensitive recruitment, (2) retention, (3) randomization requirements and practice preferences, (4) achieving treatment schedule fidelity, (5) intensive data collection, (6) the Hawthorne effect, and (7) temporal trends. The challenges experienced by EvidenceNOW grantees suggest that certain favorable real-world conditions constitute a context that increases the odds of a successful SW-CRT. An existing infrastructure can support the recruitment of many practices. Strong retention plans are needed to continue to engage sites waiting to start the intervention. Finally, study outcomes should be ones already captured in routine practice; otherwise, funders and investigators should assess the feasibility and cost of data collection.VISUAL ABSTRACT.

Inclusion of unexposed clusters improves the precision of fixed effects analysis of stepped-wedge cluster randomized trials.

Stepped-wedge cluster randomized trials (SW-CRTs) are typically analyzed using mixed effects models. The fixed effects model is a useful alternative that controls for all time-invariant cluster-level confounders and has proper control of type I error when the number of clusters is small. In principle, all clusters in SW-CRTs are designed to eventually receive the intervention, but in real-world research, some trials can end with unexposed clusters (clusters that never received the intervention), such as when a trial is terminated early based on interim analysis results. Typically, unexposed clusters are expected to contribute no information to the fixed effects intervention effect estimator and are excluded from fixed effects analyses. In this article we mathematically prove that inclusion of unexposed clusters improves the precision of the fixed effects least squares dummy variable (LSDV) intervention effect estimator, re-analyze data from a recent SW-CRT of a novel palliative care intervention containing an unexposed cluster, and evaluate the methods by simulation. We found that including unexposed clusters improves the precision of the fixed effects LSDV intervention effect estimator in both real and simulated datasets. Our simulations also reveal an increase in power and decrease in root mean square error. These improvements are present even if the assumptions of constant residual variance and period effects are violated. In the case that a SW-CRT concludes with unexposed clusters, these unexposed clusters can be included in the fixed effects LSDV analysis to improve precision, power, and root mean square error.

Estimating intervention effectiveness in trials of malaria interventions with contamination

BackgroundIn cluster randomized trials (CRTs) or stepped wedge cluster randomized trials (SWCRTs) of malaria interventions, mosquito movement leads to contamination between trial arms unless buffer zones separate the clusters. Contamination can be accounted for in the analysis, yielding an estimate of the contamination range, the distance over which contamination measurably biases the effectiveness.MethodsA previously described analysis for CRTs is extended to SWCRTs and estimates of effectiveness are provided as a function of intervention coverage. The methods are applied to two SWCRTs of malaria interventions, the SolarMal trial on the impact of mass trapping of mosquitoes with odor-baited traps and the AvecNet trial on the effect of adding pyriproxyfen to long-lasting insecticidal nets.ResultsFor the SolarMal trial, the contamination range was estimated to be 146 m (95% credible interval [0.052,,0.923] km), together with a 31.9% (95% credible interval [15.3,,45.8]%) reduction of Plasmodium infection, compared to the 30.0% reduction estimated without accounting for contamination. The estimated effectiveness had an approximately linear relationship with coverage. For the AvecNet trial, estimated contamination effects were minimal, with insufficient data from the cluster boundary regions to estimate the effectiveness as a function of coverage.ConclusionsThe contamination range in these trials of malaria interventions is much less than the distances Anopheles mosquitoes can fly. An appropriate analysis makes buffer zones unnecessary, enabling the design of more cost-efficient trials. Estimation of the contamination range requires information from the cluster boundary regions and trials should be designed to collect this.

Impact of unequal cluster sizes for GEE analyses of stepped wedge cluster randomized trials with binary outcomes.

The stepped wedge (SW) design is a type of unidirectional crossover design where cluster units switch from control to intervention condition at different prespecified time points. While a convention in study planning is to assume the cluster‐period sizes are identical, SW cluster randomized trials (SW‐CRTs) involving repeated cross‐sectional designs frequently have unequal cluster‐period sizes, which can impact the efficiency of the treatment effect estimator. In this paper, we provide a comprehensive investigation of the efficiency impact of unequal cluster sizes for generalized estimating equation analyses of SW‐CRTs, with a focus on binary outcomes as in the Washington State Expedited Partner Therapy trial. Several major distinctions between our work and existing work include the following: (i) we consider multilevel correlation structures in marginal models with binary outcomes; (ii) we study the implications of both the between‐cluster and within‐cluster imbalances in sizes; and (iii) we provide a comparison between the independence working correlation versus the true working correlation and detail the consequences of ignoring correlation estimation in SW‐CRTs with unequal cluster sizes. We conclude that the working independence assumption can lead to substantial efficiency loss and a large sample size regardless of cluster‐period size variability in SW‐CRTs, and recommend accounting for correlations in the analysis. To improve study planning, we additionally provide a computationally efficient search algorithm to estimate the sample size in SW‐CRTs accounting for unequal cluster‐period sizes, and conclude by illustrating the proposed approach in the context of the Washington State study.

Power calculation for analyses of cross-sectional stepped-wedge cluster randomized trials with binary outcomes via generalized estimating equations.

Power calculation for stepped-wedge cluster randomized trials (SW-CRTs) presents unique challenges, beyond those of standard parallel cluster randomized trials, due to the need to consider temporal within cluster correlations and background period effects. To date, power calculation methods specific to SW-CRTs have primarily been developed under a linear model. When the outcome is binary, the use of a linear model corresponds to assessing a prevalence difference; yet trial analysis often employs a nonlinear link function. We propose power calculation methods for cross-sectional SW-CRTs under a logistic model fitted by generalized estimating equations. Firstly, under an exchangeable correlation structure, we show the power based on a logistic model is lower than that from assuming a linear model in the absence of period effects. We then evaluate the impact of background prevalence changes over time on power. To allow the correlation among outcomes in the same cluster to change over time and with treatment status, we generalize the methods to more complex correlation structures. Our simulation studies demonstrate that the proposed power calculation methods perform well with the model-based variance under the true correlation structure and reveal that a working independence structure can result in substantial efficiency loss, while a working exchangeable structure performs well even when the underlying correlation structure deviates from exchangeable. An extension to our methods accounts for variable cluster sizes and reveals that unequal cluster sizes have a modest impact on power. We illustrate the approaches by application to a quality of care improvement trial for acute coronary syndrome.

Randomization-based inference for a marginal treatment effect in stepped wedge cluster randomized trials.

In a cross-sectional stepped wedge cluster randomized trial (SWT), clusters are randomized to crossover from control to intervention at different time periods and outcomes are assessed for a different set of individuals in each cluster-period. Randomization-based inference is an attractive analysis strategy for SWTs because it does not require full parametric specification of the outcome distribution or correlation structure and its validity does not rely on having a large number of clusters. Existing randomization-based approaches for SWTs, however, either focus on hypothesis testing and omit technical details on confidence interval (CI) calculation with noncontinuous outcomes, or employ weighted cluster-period summary statistics for p-value and CI calculation, which can result in suboptimal efficiency if weights do not incorporate information on varying cluster-period sizes. In this article, we propose a framework for calculating randomization-based p-values and CIs for a marginal treatment effect in SWTs by using test statistics derived from individual-level generalized linear models. We also investigate how study design features, such as stratified randomization, subsequently impact various SWT analysis methods including the proposed approach. Data from the XpertMTB/RIF tuberculosis trial are reanalyzed to illustrate our method and compare it to alternatives.

Statistical Properties of Stepped Wedge Cluster-Randomized Trials in Infectious Disease Outbreaks.

Randomized controlled trials are crucial for the evaluation of interventions such as vaccinations, but the design and analysis of these studies during infectious disease outbreaks is complicated by statistical, ethical, and logistical factors. Attempts to resolve these complexities have led to the proposal of a variety of trial designs, including individual randomization and several types of cluster randomization designs: parallel-arm, ring vaccination, and stepped wedge designs. Because of the strong time trends present in infectious disease incidence, however, methods generally used to analyze stepped wedge trials might not perform well in these settings. Using simulated outbreaks, we evaluated various designs and analysis methods, including recently proposed methods for analyzing stepped wedge trials, to determine the statistical properties of these methods. While new methods for analyzing stepped wedge trials can provide some improvement over previous methods, we find that they still lag behind parallel-arm cluster-randomized trials and individually randomized trials in achieving adequate power to detect intervention effects. We also find that these methods are highly sensitive to the weighting of effect estimates across time periods. Despite the value of new methods, stepped wedge trials still have statistical disadvantages compared with other trial designs in epidemic settings.

Novel methods for the analysis of stepped wedge cluster randomized trials.

Stepped wedge cluster randomized trials (SW-CRTs) have become increasingly popular and are used for a variety of interventions and outcomes, often chosen for their feasibility advantages. SW-CRTs must account for time trends in the outcome because of the staggered rollout of the intervention. Robust inference procedures and nonparametric analysis methods have recently been proposed to handle such trends without requiring strong parametric modeling assumptions, but these are less powerful than model-based approaches. We propose several novel analysis methods that reduce reliance on modeling assumptions while preserving some of the increased power provided by the use of mixed effects models. In one method, we use the synthetic control approach to find the best matching clusters for a given intervention cluster. Another method makes use of within-cluster crossover information to construct an overall estimator. We also consider methods that combine these approaches to further improve power. We test these methods on simulated SW-CRTs, describing scenarios in which these methods have increased power compared with existing nonparametric methods while preserving nominal validity when mixed effects models are misspecified. We also demonstrate theoretical properties of these estimators with less restrictive assumptions than mixed effects models. Finally, we propose avenues for future research on the use of these methods; motivation for such research arises from their flexibility, which allows the identification of specific causal contrasts of interest, their robustness, and the potential for incorporating covariates to further increase power. Investigators conducting SW-CRTs might well consider such methods when common modeling assumptions may not hold.

Swpermute: Permutation tests for Stepped-Wedge Cluster-Randomised Trials.

Permutation tests are useful in stepped-wedge trials to provide robust statistical tests of intervention-effect estimates. However, the Stata command permute does not produce valid tests in this setting because individual observations are not exchangeable. We introduce the swpermute command that permutes clusters to sequences to maintain exchangeability. The command provides additional functionality to aid users in performing analyses of stepped-wedge trials. In particular, we include the option "withinperiod" that performs the specified analysis separately in each period of the study with the resulting period-specific intervention-effect estimates combined as a weighted average. We also include functionality to test non-zero null hypotheses to aid the construction of confidence intervals. Examples of the application of swpermute are given using data from a trial testing the impact of a new tuberculosis diagnostic test on bacterial confirmation of a tuberculosis diagnosis.

Power calculation for cross-sectional stepped wedge cluster randomized trials with variable cluster sizes.

Standard sample size calculation formulas for stepped wedge cluster randomized trials (SW-CRTs) assume that cluster sizes are equal. When cluster sizes vary substantially, ignoring this variation may lead to an under-powered study. We investigate the relative efficiency of a SW-CRT with varying cluster sizes to equal cluster sizes, and derive variance estimators for the intervention effect that account for this variation under a mixed effects model-a commonly used approach for analyzing data from cluster randomized trials. When cluster sizes vary, the power of a SW-CRT depends on the order in which clusters receive the intervention, which is determined through randomization. We first derive a variance formula that corresponds to any particular realization of the randomized sequence and propose efficient algorithms to identify upper and lower bounds of the power. We then obtain an "expected" power based on a first-order approximation to the variance formula, where the expectation is taken with respect to all possible randomization sequences. Finally, we provide a variance formula for more general settings where only the cluster size arithmetic mean and coefficient of variation, instead of exact cluster sizes, are known in the design stage. We evaluate our methods through simulations and illustrate that the average power of a SW-CRT decreases as the variation in cluster sizes increases, and the impact is largest when the number of clusters is small.

Sample size calculation for stepped-wedge cluster-randomized trials with more than two levels of clustering.

Power and sample size calculation formulas for stepped-wedge trials with two levels (subjects within clusters) are available. However, stepped-wedge trials with more than two levels are possible. An example is the CHANGE trial which randomizes nursing homes (level 4) consisting of nursing home wards (level 3) in which nurses (level 2) are observed with respect to their hand hygiene compliance during hand hygiene opportunities (level 1) in the care of patients. We provide power and sample size methods for such trials and illustrate these in the setting of the CHANGE trial. We extend the original sample size methodology derived for stepped-wedge trials based on a random intercepts model, to accommodate more than two levels of clustering. We derive expressions that can be used to determine power and sample size for p levels of clustering in terms of the variances at each level or, alternatively, in terms of intracluster correlation coefficients. We consider different scenarios, depending on whether the same units in a particular level are repeatedly measured as a cohort sample or whether different units are measured cross-sectionally. A simple variance inflation factor is obtained that can be used to calculate power and sample size for continuous and by approximation for binary and rate outcomes. It is the product of (1) variance inflation due to the multilevel structure and (2) variance inflation due to the stepped-wedge manner of assigning interventions over time. Standard and non-standard designs (i.e. so-called "hybrid designs" and designs with more, less, or no data collection when the clusters are all in the control or are all in the intervention condition) are covered. The formulas derived enable power and sample size calculations for multilevel stepped-wedge trials. For the two-, three-, and four-level case of the standard stepped wedge, we provide programs to facilitate these calculations.

The use of permutation tests for the analysis of parallel and stepped-wedge cluster-randomized trials.

We investigate the use of permutation tests for the analysis of parallel and stepped-wedge cluster-randomized trials. Permutation tests for parallel designs with exponential family endpoints have been extensively studied. The optimal permutation tests developed for exponential family alternatives require information on intraclass correlation, a quantity not yet defined for time-to-event endpoints. Therefore, it is unclear how efficient permutation tests can be constructed for cluster-randomized trials with such endpoints. We consider a class of test statistics formed by a weighted average of pair-specific treatment effect estimates and offer practical guidance on the choice of weights to improve efficiency. We apply the permutation tests to a cluster-randomized trial evaluating the effect of an intervention to reduce the incidence of hospital-acquired infection. In some settings, outcomes from different clusters may be correlated, and we evaluate the validity and efficiency of permutation test in such settings. Lastly, we propose a permutation test for stepped-wedge designs and compare its performance with mixed-effect modeling and illustrate its superiority when sample sizes are small, the underlying distribution is skewed, or there is correlation across clusters. Copyright © 2017 John Wiley & Sons, Ltd.

Randomization inference for stepped-wedge cluster-randomized trials: An application to community-based health insurance

National health insurance schemes are generally impractical in low-income countries due to limited resources and low organizational capacity. In response to such obstacles, community-based health insurance (CBHI) schemes have emerged over the past 20 years. CBHIs are designed to reduce the financial burden generated by unanticipated treatment cost among individuals falling sick, and thus are expected to make health care more affordable. In this paper, we investigate whether CBHI schemes effectively protect individuals against large financial shocks using a stepped-wedge cluster-randomized design on data from a CBHI program rolled out in rural Burkina Faso. We investigate statistical properties of the stepped-wedge design following the parametric mixed model approach proposed by Hussey and Hughes in 2007. We find that testing for the treatment effect is generally sensitive to specification of the parametric model. For instance, if we fail to account for cluster-by-time interactions present in the data, the Type I error rate is severely inflated. We develop a more robust and efficient strategy—randomization inference. We demonstrate how to apply randomization inference to test for constant treatment effects and discuss test statistics suitable for the stepped-wedge design. Randomization inference guarantees a valid Type I error rate; simulation studies show that randomization inference test statistics also have power that is comparable to the currently used procedures that do not guarantee a valid Type I error rate. Finally, we apply our proposed method to the Burkina Faso CBHI dataset. We conclude that the insurance had limited effects on reducing the likelihood of low to moderate levels of catastrophic health expenditure, but substantially reduced the likelihood of extremely high health expenditure that exceeds half of a person’s monthly income.